Biped Balance Control by Reinforcement Learning

This work studied biped walking with single (one-leg) support and balance control using reinforcement learning. The proposed Q-learning algorithm makes a robot learn to walk without any previous knowledge of dynamics model. This balance control with single support shifts the Zero Moment Point (ZMP) of the robot to a stable region over walking sequences by means of learned gestures. Hence, the proposed method could be applied to biped walking on either plain or sloping surfaces with the help of sensory inputs. The reinforcement learning mechanism was used as the position control of the robot joints. While the robot was walking, it continuously adjusted its gaits via learning and finally formed gaits that were as stable as when it walked on the level surface. After the robot had learned to walk on an even terrain, it could learn to climb an inclined surface faster using its newly acquired knowledge. Experiments of biped walking on an even terrain and a seesaw were performed to show the validity of the proposed reinforcement learning mechanism.